According to Andrej Karpathy on Twitter, he is running a larger model on an 8×H100 setup in production for NanoChat and plans to leave the job running for an extended period. As reported by Karpathy’s post, this highlights a production-scale inference workload using NVIDIA H100 GPUs, indicating sustained high-throughput serving and stability testing for a bigger model. According to Karpathy, the configuration suggests enterprises can validate latency, throughput, and cost curves for large model deployments on H100 clusters, informing capacity planning, autoscaling, and GPU utilization strategies. As reported by the Twitter post, this scenario underscores business opportunities in model serving optimization, including quantization, tensor parallelism, and memory-efficient batching to maximize H100 occupancy.